One small, graceful monument that you might easily miss is the 4th-century B.C. Choregic Monument of Lysikrates, on Lysikratous in the Plaka, handily located just a few steps from the excellent Daphne's restaurant. This circular monument with Corinthian columns and a domed roof bears an inscription stating that Lysikrates erected it when he won the award in 334 B.C. for the best musical performance with a "chorus of boys." There's a lovely frieze showing Dionysos busily trying to turn evil pirates into friendly dolphins.
Three hills near the Acropolis deserve at least a respectful glance: the Areopagus, the Pnyx, and Filopappos. The Areopagus is the bald marble hill across from the entrance to the Acropolis; it is so slippery, despite its marble steps, that it is never an easy climb, and is positively treacherous in the rain. This makes it hard to imagine the distinguished Athenians who served on the Council and Court making their way up here to try homicide cases. Still harder to imagine is St. Paul thundering out criticisms of the Athenians for their superstitious devotion to the Unknown God from this slippery perch.
From the Areopagus and Acropolis, you can see two nearby wooded hills. The one with the monument visible on its summit is Filopappos (also known as the Hill of the Muses), and the monument is the funeral stele of the Roman consul after whom the hill is named. There are some nice walks on the wooded slopes of this hill; one nice Byzantine church, Ayios Demetrios; and the Dora Stratou Theater, where folk dances take place. If you climb to the summit (don't try this, or any wanderings here or on the Pnyx, alone at night) and face the Acropolis, you can imagine the wretched moment in 1687 when the Venetian commander Morosini shouted "Fire!" -- and his cannon shells struck the Parthenon.
The Pnyx, crowned by the Athens Observatory, is where Athens's citizen assembly met. The Pnyx, as much as any spot in Athens -- which is to say, any place in the world -- is the "birthplace of democracy." Here, for the first time in history, every citizen could vote on every matter of common importance. True, citizens did not include women, and there were far more slaves than citizens in Athens -- but that was the case in most of the world for a very long time after democracy was born in Athens.